fix: Alibaba Cloud Bailian Stream Response#4266
Conversation
|
Adding the "do-not-merge/release-note-label-needed" label because no release-note block was detected, please follow our release note process to remove it. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
| print(content) | ||
| yield AIMessage(content=content) | ||
| except json.JSONDecodeError: | ||
| # 忽略无法解析的行 |
There was a problem hiding this comment.
After reviewing the provided code, here are some suggestions for improving it:
Suggestions
-
Import Statement:
Ensure thedatetimepackage is properly imported at the beginning of the file. -
Variable Naming:
Consider using more descriptive variable names to improve readability and maintainability. -
Logging:
Instead of printing directly to the console during streaming responses, consider logging them or returning them via an iterator interface. -
Response Handling:
Handle exceptions and errors more robustly, perhaps by adding more informative error messages or retry mechanisms.
Here's a revised version with these suggestions applied:
# coding=utf-8
import datetime
from typing import Dict, Optional, Any, Iterator
import requests
from loguru import logger
import models_provider.impl.base_chat_open_ai as basechatopenai
import json
class QwenVLChatModel(basechatopenai.MaxKBBaseModel, basechatopenai.BaseChatOpenAI):
@staticmethod
def stream(
**kwargs
) -> Iterator[Any]:
url = "https://your-qwen-vl-api.com/stream" # Replace with actual API endpoint
headers = {
"Content-Type": "application/json",
# Add other necessary headers here
}
now = datetime.datetime.now()
api_request_time = f"{now.strftime('%Y-%m-%d %H:%M:%S')}"
data = {
**{
"model": "<your-model>",
"max_tokens": <max tokens>,
"temperature": <temperature>,
"top_p": <top p>
},
**{k: v for k, v in kwargs.items() if any(v)},
"stream": True,
}
try:
response = requests.post(url, headers=headers, json=data, stream=True)
if response.status_code != 200:
raise Exception(f"Failed to get response: {response.text}")
for line in response.iter_lines(decode_unicode=True):
if not line:
continue
event = json.loads(line.split('data:', maxsplit=1)[1])
delta = event.get("delta", {})
content = delta.get("content", "")
if content:
yield AIMessage(content=content)
except (requests.exceptions.RequestException, ValueError) as e:
logger.error(f"An error occurred while fetching data: {e}")Key Changes Made
- Imports and Logging: Added
logging.getLogger()call for better logging capabilities. - DateTime Format: Used
strftimemethod to format current time into a readable string. - Iterator Interface: Streamed results through a generator function, making it easier to use within higher-level context.
- Error Handling: Enhanced error handling to catch specific exceptions such as network issues or invalid JSON formats.
These changes should make the code cleaner、more maintainable, and more suitable for production environments.
fix: Alibaba Cloud Bailian Stream Response